Sharp Bounds Between Two Rényi Entropies of Distinct Positive Orders

Many axiomatic definitions of entropy, such as the Rényi entropy, of a random variable are closely related to the `α-norm of its probability distribution. This study considers probability distributions on finite sets, and examines the sharp bounds of the `β-norm with a fixed `α-norm, α 6= β, for n-dimensional probability vectors with an integer n ≥ 2. From the results, we derive the sharp bounds of the Rényi entropy of positive order β with a fixed Rényi entropy of another positive order α. As applications, we investigate sharp bounds of Ariomoto’s mutual information of order α and Gallager’s random coding exponents for uniformly focusing channels under the uniform input distribution.